Skip to content

feat: upgrade sarama from Shopify v1.30.1 to IBM v1.48.0#274

Open
Danish-sarwar1 wants to merge 3 commits intomasterfrom
feat/upgrade-sarama-ibm
Open

feat: upgrade sarama from Shopify v1.30.1 to IBM v1.48.0#274
Danish-sarwar1 wants to merge 3 commits intomasterfrom
feat/upgrade-sarama-ibm

Conversation

@Danish-sarwar1
Copy link
Copy Markdown

Upgrade Kafka client library for Kafka 4.x support. Fixes OffsetCommit v1 rejection (EOF errors) on Kafka 4.1.1. Import path: Shopify/sarama -> IBM/sarama. Go 1.25+.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 7, 2026

Important

Review skipped

Too many files!

This PR contains 299 files, which is 149 over the limit of 150.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: ca8c6632-302e-4ca5-92d6-ecd221a04697

📥 Commits

Reviewing files that changed from the base of the PR and between 9022480 and 61bd094.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (299)
  • .travis.yml
  • Makefile
  • cmd/redshiftsink/main.go
  • controllers/realtime_calculator.go
  • go.mod
  • pkg/kafka/client.go
  • pkg/kafka/consumer_group.go
  • pkg/kafka/consumer_group_sarama.go
  • pkg/kafka/producer.go
  • pkg/redshift/redshift.go
  • pkg/redshiftbatcher/batch_processor.go
  • pkg/redshiftbatcher/batcher_handler.go
  • pkg/redshiftloader/load_processor.go
  • pkg/redshiftloader/loader_handler.go
  • pkg/serializer/message.go
  • pkg/serializer/serializer.go
  • vendor/github.com/IBM/sarama/.gitignore
  • vendor/github.com/IBM/sarama/.golangci.yml
  • vendor/github.com/IBM/sarama/.pre-commit-config.yaml
  • vendor/github.com/IBM/sarama/.whitesource
  • vendor/github.com/IBM/sarama/CHANGELOG.md
  • vendor/github.com/IBM/sarama/CODE_OF_CONDUCT.md
  • vendor/github.com/IBM/sarama/CONTRIBUTING.md
  • vendor/github.com/IBM/sarama/Dockerfile.kafka
  • vendor/github.com/IBM/sarama/LICENSE.md
  • vendor/github.com/IBM/sarama/Makefile
  • vendor/github.com/IBM/sarama/README.md
  • vendor/github.com/IBM/sarama/SECURITY.md
  • vendor/github.com/IBM/sarama/Vagrantfile
  • vendor/github.com/IBM/sarama/acl_bindings.go
  • vendor/github.com/IBM/sarama/acl_create_request.go
  • vendor/github.com/IBM/sarama/acl_create_response.go
  • vendor/github.com/IBM/sarama/acl_delete_request.go
  • vendor/github.com/IBM/sarama/acl_delete_response.go
  • vendor/github.com/IBM/sarama/acl_describe_request.go
  • vendor/github.com/IBM/sarama/acl_describe_response.go
  • vendor/github.com/IBM/sarama/acl_filter.go
  • vendor/github.com/IBM/sarama/acl_types.go
  • vendor/github.com/IBM/sarama/add_offsets_to_txn_request.go
  • vendor/github.com/IBM/sarama/add_offsets_to_txn_response.go
  • vendor/github.com/IBM/sarama/add_partitions_to_txn_request.go
  • vendor/github.com/IBM/sarama/add_partitions_to_txn_response.go
  • vendor/github.com/IBM/sarama/admin.go
  • vendor/github.com/IBM/sarama/alter_client_quotas_request.go
  • vendor/github.com/IBM/sarama/alter_client_quotas_response.go
  • vendor/github.com/IBM/sarama/alter_configs_request.go
  • vendor/github.com/IBM/sarama/alter_configs_response.go
  • vendor/github.com/IBM/sarama/alter_partition_reassignments_request.go
  • vendor/github.com/IBM/sarama/alter_partition_reassignments_response.go
  • vendor/github.com/IBM/sarama/alter_user_scram_credentials_request.go
  • vendor/github.com/IBM/sarama/alter_user_scram_credentials_response.go
  • vendor/github.com/IBM/sarama/api_versions.go
  • vendor/github.com/IBM/sarama/api_versions_request.go
  • vendor/github.com/IBM/sarama/api_versions_response.go
  • vendor/github.com/IBM/sarama/async_producer.go
  • vendor/github.com/IBM/sarama/balance_strategy.go
  • vendor/github.com/IBM/sarama/broker.go
  • vendor/github.com/IBM/sarama/client.go
  • vendor/github.com/IBM/sarama/compress.go
  • vendor/github.com/IBM/sarama/config.go
  • vendor/github.com/IBM/sarama/config_resource_type.go
  • vendor/github.com/IBM/sarama/consumer.go
  • vendor/github.com/IBM/sarama/consumer_group.go
  • vendor/github.com/IBM/sarama/consumer_group_members.go
  • vendor/github.com/IBM/sarama/consumer_metadata_request.go
  • vendor/github.com/IBM/sarama/consumer_metadata_response.go
  • vendor/github.com/IBM/sarama/control_record.go
  • vendor/github.com/IBM/sarama/crc32_field.go
  • vendor/github.com/IBM/sarama/create_partitions_request.go
  • vendor/github.com/IBM/sarama/create_partitions_response.go
  • vendor/github.com/IBM/sarama/create_topics_request.go
  • vendor/github.com/IBM/sarama/create_topics_response.go
  • vendor/github.com/IBM/sarama/decompress.go
  • vendor/github.com/IBM/sarama/delete_groups_request.go
  • vendor/github.com/IBM/sarama/delete_groups_response.go
  • vendor/github.com/IBM/sarama/delete_offsets_request.go
  • vendor/github.com/IBM/sarama/delete_offsets_response.go
  • vendor/github.com/IBM/sarama/delete_records_request.go
  • vendor/github.com/IBM/sarama/delete_records_response.go
  • vendor/github.com/IBM/sarama/delete_topics_request.go
  • vendor/github.com/IBM/sarama/delete_topics_response.go
  • vendor/github.com/IBM/sarama/describe_client_quotas_request.go
  • vendor/github.com/IBM/sarama/describe_client_quotas_response.go
  • vendor/github.com/IBM/sarama/describe_cluster_request.go
  • vendor/github.com/IBM/sarama/describe_cluster_response.go
  • vendor/github.com/IBM/sarama/describe_configs_request.go
  • vendor/github.com/IBM/sarama/describe_configs_response.go
  • vendor/github.com/IBM/sarama/describe_groups_request.go
  • vendor/github.com/IBM/sarama/describe_groups_response.go
  • vendor/github.com/IBM/sarama/describe_log_dirs_request.go
  • vendor/github.com/IBM/sarama/describe_log_dirs_response.go
  • vendor/github.com/IBM/sarama/describe_user_scram_credentials_request.go
  • vendor/github.com/IBM/sarama/describe_user_scram_credentials_response.go
  • vendor/github.com/IBM/sarama/dev.yml
  • vendor/github.com/IBM/sarama/docker-compose.yml
  • vendor/github.com/IBM/sarama/elect_leaders_request.go
  • vendor/github.com/IBM/sarama/elect_leaders_response.go
  • vendor/github.com/IBM/sarama/election_type.go
  • vendor/github.com/IBM/sarama/encoder_decoder.go
  • vendor/github.com/IBM/sarama/end_txn_request.go
  • vendor/github.com/IBM/sarama/end_txn_response.go
  • vendor/github.com/IBM/sarama/entrypoint.sh
  • vendor/github.com/IBM/sarama/errors.go
  • vendor/github.com/IBM/sarama/fetch_request.go
  • vendor/github.com/IBM/sarama/fetch_response.go
  • vendor/github.com/IBM/sarama/find_coordinator_request.go
  • vendor/github.com/IBM/sarama/find_coordinator_response.go
  • vendor/github.com/IBM/sarama/gssapi_kerberos.go
  • vendor/github.com/IBM/sarama/heartbeat_request.go
  • vendor/github.com/IBM/sarama/heartbeat_response.go
  • vendor/github.com/IBM/sarama/incremental_alter_configs_request.go
  • vendor/github.com/IBM/sarama/incremental_alter_configs_response.go
  • vendor/github.com/IBM/sarama/init_producer_id_request.go
  • vendor/github.com/IBM/sarama/init_producer_id_response.go
  • vendor/github.com/IBM/sarama/interceptors.go
  • vendor/github.com/IBM/sarama/join_group_request.go
  • vendor/github.com/IBM/sarama/join_group_response.go
  • vendor/github.com/IBM/sarama/kerberos_client.go
  • vendor/github.com/IBM/sarama/leave_group_request.go
  • vendor/github.com/IBM/sarama/leave_group_response.go
  • vendor/github.com/IBM/sarama/length_field.go
  • vendor/github.com/IBM/sarama/list_groups_request.go
  • vendor/github.com/IBM/sarama/list_groups_response.go
  • vendor/github.com/IBM/sarama/list_partition_reassignments_request.go
  • vendor/github.com/IBM/sarama/list_partition_reassignments_response.go
  • vendor/github.com/IBM/sarama/message.go
  • vendor/github.com/IBM/sarama/message_set.go
  • vendor/github.com/IBM/sarama/metadata.go
  • vendor/github.com/IBM/sarama/metadata_request.go
  • vendor/github.com/IBM/sarama/metadata_response.go
  • vendor/github.com/IBM/sarama/metrics.go
  • vendor/github.com/IBM/sarama/mockbroker.go
  • vendor/github.com/IBM/sarama/mockkerberos.go
  • vendor/github.com/IBM/sarama/mockresponses.go
  • vendor/github.com/IBM/sarama/offset_commit_request.go
  • vendor/github.com/IBM/sarama/offset_commit_response.go
  • vendor/github.com/IBM/sarama/offset_fetch_request.go
  • vendor/github.com/IBM/sarama/offset_fetch_response.go
  • vendor/github.com/IBM/sarama/offset_manager.go
  • vendor/github.com/IBM/sarama/offset_request.go
  • vendor/github.com/IBM/sarama/offset_response.go
  • vendor/github.com/IBM/sarama/packet_decoder.go
  • vendor/github.com/IBM/sarama/packet_encoder.go
  • vendor/github.com/IBM/sarama/partitioner.go
  • vendor/github.com/IBM/sarama/prep_encoder.go
  • vendor/github.com/IBM/sarama/produce_request.go
  • vendor/github.com/IBM/sarama/produce_response.go
  • vendor/github.com/IBM/sarama/produce_set.go
  • vendor/github.com/IBM/sarama/quota_types.go
  • vendor/github.com/IBM/sarama/real_decoder.go
  • vendor/github.com/IBM/sarama/real_encoder.go
  • vendor/github.com/IBM/sarama/record.go
  • vendor/github.com/IBM/sarama/record_batch.go
  • vendor/github.com/IBM/sarama/records.go
  • vendor/github.com/IBM/sarama/request.go
  • vendor/github.com/IBM/sarama/response_header.go
  • vendor/github.com/IBM/sarama/sarama.go
  • vendor/github.com/IBM/sarama/sasl_authenticate_request.go
  • vendor/github.com/IBM/sarama/sasl_authenticate_response.go
  • vendor/github.com/IBM/sarama/sasl_handshake_request.go
  • vendor/github.com/IBM/sarama/sasl_handshake_response.go
  • vendor/github.com/IBM/sarama/scram_formatter.go
  • vendor/github.com/IBM/sarama/server.properties
  • vendor/github.com/IBM/sarama/sockopt_other.go
  • vendor/github.com/IBM/sarama/sockopt_posix.go
  • vendor/github.com/IBM/sarama/sticky_assignor_user_data.go
  • vendor/github.com/IBM/sarama/sync_group_request.go
  • vendor/github.com/IBM/sarama/sync_group_response.go
  • vendor/github.com/IBM/sarama/sync_producer.go
  • vendor/github.com/IBM/sarama/timestamp.go
  • vendor/github.com/IBM/sarama/transaction_manager.go
  • vendor/github.com/IBM/sarama/txn_offset_commit_request.go
  • vendor/github.com/IBM/sarama/txn_offset_commit_response.go
  • vendor/github.com/IBM/sarama/utils.go
  • vendor/github.com/IBM/sarama/version.go
  • vendor/github.com/IBM/sarama/zstd.go
  • vendor/github.com/Shopify/sarama/.gitignore
  • vendor/github.com/Shopify/sarama/.golangci.yml
  • vendor/github.com/Shopify/sarama/CHANGELOG.md
  • vendor/github.com/Shopify/sarama/Makefile
  • vendor/github.com/Shopify/sarama/compress.go
  • vendor/github.com/Shopify/sarama/consumer_group_members.go
  • vendor/github.com/Shopify/sarama/create_topics_response.go
  • vendor/github.com/Shopify/sarama/decompress.go
  • vendor/github.com/Shopify/sarama/delete_groups_request.go
  • vendor/github.com/Shopify/sarama/delete_groups_response.go
  • vendor/github.com/Shopify/sarama/delete_topics_request.go
  • vendor/github.com/Shopify/sarama/describe_groups_request.go
  • vendor/github.com/Shopify/sarama/describe_groups_response.go
  • vendor/github.com/Shopify/sarama/docker-compose.yml
  • vendor/github.com/Shopify/sarama/end_txn_response.go
  • vendor/github.com/Shopify/sarama/errors.go
  • vendor/github.com/Shopify/sarama/heartbeat_request.go
  • vendor/github.com/Shopify/sarama/heartbeat_response.go
  • vendor/github.com/Shopify/sarama/init_producer_id_request.go
  • vendor/github.com/Shopify/sarama/init_producer_id_response.go
  • vendor/github.com/Shopify/sarama/join_group_response.go
  • vendor/github.com/Shopify/sarama/leave_group_request.go
  • vendor/github.com/Shopify/sarama/leave_group_response.go
  • vendor/github.com/Shopify/sarama/list_groups_request.go
  • vendor/github.com/Shopify/sarama/list_groups_response.go
  • vendor/github.com/Shopify/sarama/metadata_request.go
  • vendor/github.com/Shopify/sarama/metadata_response.go
  • vendor/github.com/Shopify/sarama/metrics.go
  • vendor/github.com/Shopify/sarama/request.go
  • vendor/github.com/Shopify/sarama/sasl_authenticate_response.go
  • vendor/github.com/Shopify/sarama/sync_group_request.go
  • vendor/github.com/Shopify/sarama/sync_group_response.go
  • vendor/github.com/Shopify/sarama/version.go
  • vendor/github.com/Shopify/sarama/zstd.go
  • vendor/github.com/eapache/go-resiliency/breaker/README.md
  • vendor/github.com/eapache/go-resiliency/breaker/breaker.go
  • vendor/github.com/eapache/go-xerial-snappy/.travis.yml
  • vendor/github.com/eapache/go-xerial-snappy/README.md
  • vendor/github.com/eapache/go-xerial-snappy/fuzz.go
  • vendor/github.com/eapache/go-xerial-snappy/snappy.go
  • vendor/github.com/hashicorp/go-uuid/LICENSE
  • vendor/github.com/jcmturner/gokrb5/v8/client/TGSExchange.go
  • vendor/github.com/jcmturner/gokrb5/v8/client/network.go
  • vendor/github.com/jcmturner/gokrb5/v8/client/session.go
  • vendor/github.com/jcmturner/gokrb5/v8/config/krb5conf.go
  • vendor/github.com/jcmturner/gokrb5/v8/credentials/ccache.go
  • vendor/github.com/jcmturner/gokrb5/v8/keytab/keytab.go
  • vendor/github.com/jcmturner/gokrb5/v8/types/Authenticator.go
  • vendor/github.com/klauspost/compress/.gitignore
  • vendor/github.com/klauspost/compress/.goreleaser.yml
  • vendor/github.com/klauspost/compress/README.md
  • vendor/github.com/klauspost/compress/SECURITY.md
  • vendor/github.com/klauspost/compress/flate/deflate.go
  • vendor/github.com/klauspost/compress/flate/dict_decoder.go
  • vendor/github.com/klauspost/compress/flate/fast_encoder.go
  • vendor/github.com/klauspost/compress/flate/huffman_bit_writer.go
  • vendor/github.com/klauspost/compress/flate/huffman_code.go
  • vendor/github.com/klauspost/compress/flate/huffman_sortByFreq.go
  • vendor/github.com/klauspost/compress/flate/huffman_sortByLiteral.go
  • vendor/github.com/klauspost/compress/flate/inflate.go
  • vendor/github.com/klauspost/compress/flate/inflate_gen.go
  • vendor/github.com/klauspost/compress/flate/level1.go
  • vendor/github.com/klauspost/compress/flate/level2.go
  • vendor/github.com/klauspost/compress/flate/level3.go
  • vendor/github.com/klauspost/compress/flate/level4.go
  • vendor/github.com/klauspost/compress/flate/level5.go
  • vendor/github.com/klauspost/compress/flate/level6.go
  • vendor/github.com/klauspost/compress/flate/matchlen_generic.go
  • vendor/github.com/klauspost/compress/flate/regmask_amd64.go
  • vendor/github.com/klauspost/compress/flate/regmask_other.go
  • vendor/github.com/klauspost/compress/flate/stateless.go
  • vendor/github.com/klauspost/compress/flate/token.go
  • vendor/github.com/klauspost/compress/fse/bitwriter.go
  • vendor/github.com/klauspost/compress/fse/compress.go
  • vendor/github.com/klauspost/compress/fse/decompress.go
  • vendor/github.com/klauspost/compress/gzip/gunzip.go
  • vendor/github.com/klauspost/compress/gzip/gzip.go
  • vendor/github.com/klauspost/compress/huff0/bitreader.go
  • vendor/github.com/klauspost/compress/huff0/bitwriter.go
  • vendor/github.com/klauspost/compress/huff0/bytereader.go
  • vendor/github.com/klauspost/compress/huff0/compress.go
  • vendor/github.com/klauspost/compress/huff0/decompress.go
  • vendor/github.com/klauspost/compress/huff0/decompress_amd64.go
  • vendor/github.com/klauspost/compress/huff0/decompress_amd64.s
  • vendor/github.com/klauspost/compress/huff0/decompress_generic.go
  • vendor/github.com/klauspost/compress/huff0/huff0.go
  • vendor/github.com/klauspost/compress/internal/cpuinfo/cpuinfo.go
  • vendor/github.com/klauspost/compress/internal/cpuinfo/cpuinfo_amd64.go
  • vendor/github.com/klauspost/compress/internal/cpuinfo/cpuinfo_amd64.s
  • vendor/github.com/klauspost/compress/internal/le/le.go
  • vendor/github.com/klauspost/compress/internal/le/unsafe_disabled.go
  • vendor/github.com/klauspost/compress/internal/le/unsafe_enabled.go
  • vendor/github.com/klauspost/compress/internal/race/norace.go
  • vendor/github.com/klauspost/compress/internal/race/race.go
  • vendor/github.com/klauspost/compress/internal/snapref/decode.go
  • vendor/github.com/klauspost/compress/internal/snapref/encode.go
  • vendor/github.com/klauspost/compress/internal/snapref/encode_other.go
  • vendor/github.com/klauspost/compress/s2/.gitignore
  • vendor/github.com/klauspost/compress/s2/LICENSE
  • vendor/github.com/klauspost/compress/s2/README.md
  • vendor/github.com/klauspost/compress/s2/decode.go
  • vendor/github.com/klauspost/compress/s2/decode_amd64.s
  • vendor/github.com/klauspost/compress/s2/decode_arm64.s
  • vendor/github.com/klauspost/compress/s2/decode_asm.go
  • vendor/github.com/klauspost/compress/s2/decode_other.go
  • vendor/github.com/klauspost/compress/s2/dict.go
  • vendor/github.com/klauspost/compress/s2/encode.go
  • vendor/github.com/klauspost/compress/s2/encode_all.go
  • vendor/github.com/klauspost/compress/s2/encode_amd64.go
  • vendor/github.com/klauspost/compress/s2/encode_best.go
  • vendor/github.com/klauspost/compress/s2/encode_better.go
  • vendor/github.com/klauspost/compress/s2/encode_go.go
  • vendor/github.com/klauspost/compress/s2/encodeblock_amd64.go
  • vendor/github.com/klauspost/compress/s2/encodeblock_amd64.s
  • vendor/github.com/klauspost/compress/s2/index.go
  • vendor/github.com/klauspost/compress/s2/lz4convert.go
  • vendor/github.com/klauspost/compress/s2/lz4sconvert.go
  • vendor/github.com/klauspost/compress/s2/reader.go
  • vendor/github.com/klauspost/compress/s2/s2.go
  • vendor/github.com/klauspost/compress/s2/writer.go
  • vendor/github.com/klauspost/compress/s2sx.mod
  • vendor/github.com/klauspost/compress/snappy/AUTHORS
  • vendor/github.com/klauspost/compress/snappy/CONTRIBUTORS

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/upgrade-sarama-ibm

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

- Replace all github.com/Shopify/sarama imports with github.com/IBM/sarama
- Fix error comparisons to use errors.Is() pattern
- Adds Kafka 4.x protocol support (fixes OffsetCommit v1 rejection)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Danish-sarwar1 Danish-sarwar1 force-pushed the feat/upgrade-sarama-ibm branch from 7561b8e to f453c44 Compare May 7, 2026 18:10
Danish Sarwar added 2 commits May 8, 2026 15:49
go.mod requires go 1.25.0 after the IBM/sarama upgrade. Bump the
build-container image and Travis host Go version to match, and
reformat two doc comments to satisfy Go 1.19+ gofmt rules.
- LeaderElectionID '854ae6e3.' -> 'redshiftsink-operator-leader' for
  readability. Note: rolling deploys across this change will briefly
  hold two distinct leases.
- realtime_calculator.go: drop unused SQL fragment accidentally
  appended after the package's last function.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant